Skip to content

feat(ai): hold auto-applied repairs to the previous version's hosts (+ scan specs) - #663

Merged
fclairamb merged 16 commits into
mainfrom
security/scan-specs-2026-10-24
Oct 25, 2026
Merged

fclairamb merged 16 commits into
mainfrom
security/scan-specs-2026-10-24

Conversation

@fclairamb

@fclairamb fclairamb commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Specs filed from the daily security scans. Each spec is the tracker for its finding.

This branch also carries the maintainer's R1 fix (08c3aa21 and follow-ups), so it is no longer specs-only.

Open (added by this PR)

  • S1 (medium): specs/todos/2026-10-25-01-ai-repair-grant-dropped-by-reference-resolution.md. The R1 repair grant is compared against the script and env after ${var:} / ${param:} / ${env:} references are resolved. Any reference in an auto-repaired script or env therefore silently drops the host allowlist on scheduled runs. It is a bypass of the R1 fix on this branch.

Earlier scan specs on this branch (implemented, in specs/done/2026/10/)

  • R1 (medium): 2026-10-24-01-ai-repair-auto-apply-partial-runtime-host.md. repair: auto applied a candidate that kept a literal host and added one built at runtime. Fixed on this branch; see S1 for a gap.
  • Q1 (medium): 2026-10-23-02-ai-loop-secret-relay-through-model.md. AI loops relayed an encoded secret through the model.
  • P1 (medium): 2026-10-22-01-ai-script-secret-host-guard-runtime.md. AI runs with real secrets were only host-checked by regex.
  • K1 (medium): 2026-10-20-01-sender-response-body-caps.md. Notification senders and RabbitMQ read responses uncapped.
  • J1 (medium): 2026-10-19-01-a2a-response-size-cap.md. A2A responses were read uncapped.
  • I1 (low): 2026-10-18-01-plugin-schema-file-ref.md. Plugin schema $ref could load local files.
  • G1/G2 (low): 2026-10-17-01-postgres-check-connection-config.md. Postgres conninfo escaping and fallback to the worker's PG* env.
  • F1: 2026-10-16-15-pat-cache-revocation.md. The PAT cache outlived revocation.
  • E1: 2026-10-16-14-totp-on-federated-sign-in.md. No TOTP on federated sign-in.
  • E2: 2026-10-16-13-atomic-auth-attempt-counters.md. Auth attempt counters were not atomic.
  • H3: 2026-10-16-12-mcp-tokens-rejected-on-rest.md. MCP OAuth tokens worked on REST.
  • H2: 2026-10-16-11-slack-signature-fail-closed.md. Slack signature check failed open.
  • T1 / M2 (high): 2026-10-16-10-discovery-scans-egress-guard.md. Discovery scans were outside the egress policy.
  • N4: 2026-10-16-09-js-check-nested-docker-guard.md. A JS docker sub-check bypassed the gate.
  • N3 (high): 2026-10-16-08-status-page-components-cross-org.md. Status-page components could be edited across orgs.
  • D2/D3 (high): 2026-10-16-07-guard-status-subscriber-and-publication-webhooks.md. Subscriber and publication webhooks were not guarded.
  • C3: 2026-10-16-06-accept-invite-existing-account.md. Accepting an invite logged in as an existing account.
  • C1: 2026-10-16-05-kubeconfig-exec-and-incluster.md. Kubeconfig exec and inCluster were allowed.
  • N2: 2026-10-16-04-mysql-dsn-injection.md. MySQL DSN injection.
  • N1: 2026-10-16-03-env-secret-reference-allowlist.md. No allowlist for ${env:} references.
  • C2/H1: 2026-10-16-02-token-purpose-separation.md. Purpose tokens were accepted as access tokens.
  • D1: 2026-10-16-01-microsoft-signin-unverified-email-linking.md. Microsoft nOAuth email linking.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.94313% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.17%. Comparing base (15de67a) to head (9d305e3).

Files with missing lines Patch % Lines
server/internal/aichecks/guards.go 92.68% 6 Missing ⚠️
server/internal/aichecks/revoke.go 86.66% 4 Missing ⚠️
server/internal/checkers/checkjs/config/ai.go 88.00% 3 Missing ⚠️
server/internal/aichecks/repair.go 88.88% 2 Missing ⚠️
server/internal/checkers/checkjs/config/config.go 95.83% 1 Missing ⚠️
server/internal/handlers/incidents/ai_repair.go 87.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #663      +/-   ##
==========================================
- Coverage   88.17%   88.17%   -0.01%     
==========================================
  Files        1288     1289       +1     
  Lines      131355   131553     +198     
==========================================
+ Hits       115827   116001     +174     
- Misses      15519    15543      +24     
  Partials        9        9              
Files with missing lines Coverage Δ
server/internal/checkers/checkjs/checker.go 91.81% <100.00%> (+0.47%) ⬆️
server/internal/db/models/event.go 100.00% <ø> (ø)
server/internal/handlers/incidents/service.go 87.95% <ø> (+0.29%) ⬆️
server/internal/handlers/system/service.go 76.36% <ø> (ø)
server/internal/checkers/checkjs/config/config.go 91.22% <95.83%> (+1.22%) ⬆️
server/internal/handlers/incidents/ai_repair.go 69.56% <87.50%> (+4.85%) ⬆️
server/internal/aichecks/repair.go 87.22% <88.88%> (-0.18%) ⬇️
server/internal/checkers/checkjs/config/ai.go 89.71% <88.00%> (+0.30%) ⬆️
server/internal/aichecks/revoke.go 86.66% <86.66%> (ø)
server/internal/aichecks/guards.go 90.77% <92.68%> (+1.97%) ⬆️

... and 22 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

fclairamb and others added 12 commits October 24, 2026 11:24
…annot take it

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A `repair: auto` candidate that kept the previous literal host and added a
branch to a host built at runtime passed every gate and was applied. Its
scheduled runs had no host allowlist, so the branch could send a secret to
any public host (security scan 2026-10-24, finding R1).

- applyRepair stores the hosts the previous version reached in the ai block
  (`repair_hosts`), bound to the applied script and env (`repair_grant`).
- Every run of that version with real secrets is held to them. A refusal
  fails the run with an error result flagged `aiRepairHostRefused`.
- Such a result sets the check back to `repair: propose` and records a
  `check.ai_repair_revoked` event, instead of queueing a repair.
- Editing the script or env drops the grant (js NormalizeConfig).
- `repair: auto` now proposes any candidate with a network call whose host
  its text does not show, or more network calls than before (hiddenHost,
  replaces hostsHidden).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eeps a readable host and adds one built at runtime

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@fclairamb fclairamb changed the title chore(specs): security scan findings feat(ai): hold auto-applied repairs to the previous version's hosts (+ scan specs) Oct 25, 2026
@fclairamb
fclairamb enabled auto-merge (squash) October 25, 2026 16:15
@fclairamb
fclairamb merged commit 37966e2 into main Oct 25, 2026
33 checks passed
@fclairamb
fclairamb deleted the security/scan-specs-2026-10-24 branch October 25, 2026 16:39
fclairamb added a commit that referenced this pull request Oct 26, 2026
)

Specs filed from the daily security scans, and the fix for the one still
open.

## What changed

- **U1 (low)**:
`specs/done/2026/10/2026-10-26-02-ai-repair-hold-skips-param-only-secrets.md`.
Every scheduled run of a `repair: auto` version is now held to its
granted hosts, whether its credentials sit in `secrets` or come from a
`${param:…}` / `${env:…}` reference. Before, a check with no `secrets`
value ran unheld with the resolved value. Tests cover the checker, the
worker claim path and the agent path, each with a positive control. Docs
updated. Batch gate green (`make build-backend lint-back test`).

## Decided by autopilot

- Nothing: the spec had no open questions.

## Earlier scan specs (implemented, in `specs/done/2026/10/`)


- **S1 (medium)**:
`2026-10-25-01-ai-repair-grant-dropped-by-reference-resolution.md`. The
repair grant was compared after `${…}` references were resolved. Fixed
in #678; see U1 for a remaining gap.
- **R1 (medium)**:
`2026-10-24-01-ai-repair-auto-apply-partial-runtime-host.md`. `repair:
auto` applied a candidate that kept a literal host and added one built
at runtime. Fixed in #663.
- **Q1 (medium)**:
`2026-10-23-02-ai-loop-secret-relay-through-model.md`. AI loops relayed
an encoded secret through the model.
- **P1 (medium)**:
`2026-10-22-01-ai-script-secret-host-guard-runtime.md`. AI runs with
real secrets were only host-checked by regex.
- **K1 (medium)**: `2026-10-20-01-sender-response-body-caps.md`.
Notification senders and RabbitMQ read responses uncapped.
- **J1 (medium)**: `2026-10-19-01-a2a-response-size-cap.md`. A2A
responses were read uncapped.
- **I1 (low)**: `2026-10-18-01-plugin-schema-file-ref.md`. Plugin schema
`$ref` could load local files.
- **G1/G2 (low)**: `2026-10-17-01-postgres-check-connection-config.md`.
Postgres conninfo escaping and fallback to the worker's `PG*` env.
- **F1**: `2026-10-16-15-pat-cache-revocation.md`. The PAT cache
outlived revocation.
- **E1**: `2026-10-16-14-totp-on-federated-sign-in.md`. No TOTP on
federated sign-in.
- **E2**: `2026-10-16-13-atomic-auth-attempt-counters.md`. Auth attempt
counters were not atomic.
- **H3**: `2026-10-16-12-mcp-tokens-rejected-on-rest.md`. MCP OAuth
tokens worked on REST.
- **H2**: `2026-10-16-11-slack-signature-fail-closed.md`. Slack
signature check failed open.
- **T1 / M2 (high)**: `2026-10-16-10-discovery-scans-egress-guard.md`.
Discovery scans were outside the egress policy.
- **N4**: `2026-10-16-09-js-check-nested-docker-guard.md`. A JS `docker`
sub-check bypassed the gate.
- **N3 (high)**: `2026-10-16-08-status-page-components-cross-org.md`.
Status-page components could be edited across orgs.
- **D2/D3 (high)**:
`2026-10-16-07-guard-status-subscriber-and-publication-webhooks.md`.
Subscriber and publication webhooks were not guarded.
- **C3**: `2026-10-16-06-accept-invite-existing-account.md`. Accepting
an invite logged in as an existing account.
- **C1**: `2026-10-16-05-kubeconfig-exec-and-incluster.md`. Kubeconfig
`exec` and `inCluster` were allowed.
- **N2**: `2026-10-16-04-mysql-dsn-injection.md`. MySQL DSN injection.
- **N1**: `2026-10-16-03-env-secret-reference-allowlist.md`. No
allowlist for `${env:}` references.
- **C2/H1**: `2026-10-16-02-token-purpose-separation.md`. Purpose tokens
were accepted as access tokens.
- **D1**: `2026-10-16-01-microsoft-signin-unverified-email-linking.md`.
Microsoft nOAuth email linking.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant